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Background / Context: 

Description of prior research and its intellectual context. 

The number of students that live in families where a language other than English is 
spoken has risen relative to the English-only households in the United States over the last 25 
years. These students face the dual challenge of mastering English while acquiring academic 
skills and knowledge. 

The education of these students has been shaped by several national- and state- level legal 
decisions of the past that have required schools to take action by providing services to help 
linguistic minority students overcome language barriers that impede their equal participation. 
Those services most often take the form of either an “English-immersion” approach, in which 
students receive all instruction in English, or employ a “bilingual model” where students are 
initially taught in some combination of English and their native language and eventually transfer 
to English-only classrooms. The question of which model of instruction is more “effective” has 
been notoriously difficult to answer and remains an open and controversial debate. 

The literature is not conclusive with respect to which instructional program better serves 
the needs of English learners. This may be in part due to the fact that of the hundreds of small- 
scale studies that have been conducted to date, very few statistically account for the problems of 
selection into bilingual education. Rossell & Baker (1996) conducted a meta-analysis of 300 
evaluation studies and found that only 25 percent of them used an acceptable methodology (i.e., 
they had a treatment and control group and appropriately controlled for pre-treatment differences 
across groups). Slavin & Cheung (2005), in a synthesis of the research also find similar 
methodological issues, with many of these studies lacking appropriate comparison groups or 
involving treatment durations of less than one year. 

A number of recent studies have used either experimental or quasi-experimental research 
designs to overcome the issues mentioned above and measure the causal link between language 
of instruction and academic performance (Slavin et al, 2011; Matsudaira, 2005; Chin, Daysal, & 
Imberman, 2011). While these studies have focused on the evaluation of different instructional 
programs to educate English Eearners (EEs), the major focus has been on how differences in the 
language of instruction affect the academic achievement of EEs, rather than understanding the 
mechanisms through which peers and classroom composition could affect their outcomes. The 
literature on peer composition has played an important role in policy debates concerning ability 
tracking, school desegregation, and affirmative action programs, but has not been an important 
aspect of the program evaluation literature on instructional programs for English learners.' This 
study seeks to fill in this gap by showing that the composition of the class is an important aspect 
that should be taken into consideration when analyzing the effects of different instructional 
programs for EEs. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 


' This literature dates back to the Coleman Report and has been plagued by identification and measurement issues. 
Most recent and rigorous studies have found positive peer effects on achievement growth (e.g. Hanushek et al. 
(2003), Hoxby (2000), Sacerdote (2001), and Ding & Lehrer (2006)). 


SREE Spring 2014 Conference Abstract Template 


1 



In this article we take advantage of a policy passed in California that changed the default 
instructional program for ELs (described later). Prior to 1998, most ELs were placed into 
bilingual education. This state-level legislation forced schools to move students to move to a 
different instructional setting -a change that would not had been chosen otherwise — providing a 
natural experiment opportunity to evaluate the effect of bilingual education versus English 
immersion on the academic achievement of EEs. We also analyze how changes in the 
composition of the classroom affect the academic achievement of students. 

Setting: 

Description of the research location. 

With a very diverse student population, California’s educational system provides a good 
opportunity to study the impact of instructional programs geared towards aiding students to 
overcome the language barrier. About a quarter of the students enrolled in California’s schools 
are classified as EEs. Our study focuses on the largest school district in California, the Eos 
Angeles Unified School District (EAUSD), where approximately 40% of its students were 
classified as EEs when Proposition 227 was passed in 1998. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

We use a EAUSD student-level panel database from 1996-97 through 2001-02 (two years 
before the passage of the proposition to four years after). We have information on students’ test 
scores (standardized to have mean zero and variance one), instructional programs (for the 
purpose of this analysis, several programs were grouped together into two main groups: bilingual 
and English immersion), demographic characteristics, and language classification. This database 
also includes an identifier for the teacher so that we are able to identify the classroom 
characteristics/composition that a given student is exposed to over time. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

In this Study we analyze how different instructional settings (bilingual and English 
immersion) affect the academic achievement of EEs, by taking into consideration the changes in 
the composition of the classroom. We use Proposition 227 as a natural experiment.^ This 
proposition was passed in June of 1998, mandating that beginning in the school year 1998-99, all 
EEs in California are to be placed in structured English immersion for a period “not normally 
intended to exceed one year”, then be transferred to mainstream classrooms taught 
overwhelmingly in English. As a result, this proposition significantly altered the ways in which 
the state’s EEs were to be taught. 

It should be noted that the compliance with the proposition was very high. This is 
evidenced by the sharp decline in the enrollment of Spanish EEs in bilingual programs (Eigure 
1). One possible factor could be the strong language used in the proposition regarding the failure 
to comply with the implementation of the law. The law states that any elected official, public 
school teacher or administrator, who willfully and repeatedly refuses to implement the terms of 
the law, “may be held personally liable for fees and actual damages by the child’s parents or 
legal guardian.” It is important to point out that the reason why bilingual education enrollment 
did not fall to zero after the passage of the proposition is the parental waiver exception provision. 


^ Full text of Proposition 227 can be found here; http://primary98.sos.ca.gov/VoterGuide/Propositions/227text.htm 
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This provision allows parents to request bilingual education for their children with the approval 
of both the teacher and principal. In our identification strategy we take into account the fact that 
the change in the percentage of students enrolled in bilingual education is likely to be 
endogenous. 

(please insert figure 1 here) 

Research Design: 

Description of the research design. 

We exploit the sharpness of the change in bilingual education enrollment in the year the 
proposition was passed for our identification strategy. Our main assumption is that with the 
passage of the proposition schools were forced to make large changes in the share of ELs 
enrolled in bilingual education and that no other large change occurred in these schools that 
could be correlated with the share of ELs in bilingual education during this time. However, we 
need to take into account the fact that the change in bilingual enrollment is not completely 
exogenous. Eor example, some schools could have reacted to the ban by poorly implementing 
English immersion programs, or not eliminating bilingual education by seeking parental waivers. 
To remedy this issue we use an instrumental variables technique (with mechanical 
implementation of the policy as an instrument). We regress the change in academic achievement 
(English and Math) for both ELs and native English speakers (EOs), before and after the passage 
of the proposition, as a function of the instrumented change in the share of ELs enrolled in 
bilingual education at the school-level. 


Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

As previously described, we use administrative student- level panel data collected by 
LAUSD. We focus on the change in students’ academic achievement between 1997-98 and 
1999-00.^ Eormally: We estimate our models within schools. In addition, in order to take into 
account the non-random sorting of teachers into classrooms, in some specifications we 
incorporate teachers’ fixed effects. 


.ysub.g.l 


= + X,X + X,Y + Si,s,c 


( 1 ) 


S'Ltb CJ I 

Where the AT^ ^ ^ represents the change in academic achievement for student i, in school 5, and 
classroom c. We estimate this model for English and Math standardized test scores (sub), across 
different grade levels (g) as well as separately for ELs and EOs (/). A%Bilg represents the 
instrumented change in the share of ELs in bilingual education in school s (this is our measure of 
the extent that a school implements Proposition 227, a negative value indicates that a smaller 
percentage of ELs were placed into bilingual education after the passage of Proposition 227 
compared to before). We estimate this model with observable school and classroom controls 
(Xg and Xf., respectively). We also run specifications with school or classroom fixed effects. 

In addition, we incorporate a measure of the change in exposure of ELs to native English 
speakers (A%EO) in the classroom, as well as an interaction between our treatment and this 
exposure measure. Eormally: 


^ We also estimate the change in academic achievement between 1997-98 and 1998-99, 1997-98 and 2000-01, as 
well as 1997-98 and 2001-02. 
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^cV + ^i,s,c ( 2 ) 

Findings / Results: 

Description of the main findings with specific details. 

Tables 1-4 summarize our main findings. We present the results for the change between 
97-1998 and 1999-00 school years, although our results are robust to different time periods. 
Across all these tables, column (1) shows OLS results without any controls. Column (2) presents 
our 2SLS estimates without any controls as well. Columns (3) - (5) show our 2SLS results of 
Equation (1) with and without school and classroom controls (or fixed effects). Columns (6) - 
(8) show the results of Equation (2) estimated by 2SES as well (with and without controls/fixed 
effects). 

The first panel in Table 1 presents EE math achievement results for a pooled regression 
for grades 2-3, grades 4-6 are shown in the second panel. We find little effect of the proposition 
on earlier grades on math results once we control for class and school characteristics. We, 
however, find that EE students in grades 4-6 experience declines in math scores as a result of the 
switch from Bilingual to English immersion after controlling for class and school characteristics 
as well as the change in the share of EOs in the classroom (remember a negative A%Bilg 
variable is what would be observed in most schools after Proposition 227). We also generally see 
a beneficial effect of placing EEs into classrooms with more EOs on their math scores. 
Interpreting the coefficients on the interaction terms, we find that EE math scores rose after the 
switch from bilingual to English immersion when that was combined with an increased number 
of EOs in the classroom (though this effect is generally insignificant). 

Table 2 shows reading results for EEs. We find a small positive effect on grades 2-3. We 
also find that while increasing the percentage of EOs in the classroom helps; it does not help 
when combined with a change in educational program. The reading scores for EEs in grades 4-6 
decreased after the passage of the proposition. While we see an increase in reading scores after 
an increase in the percentage of EOs in the classroom, we do not see that same increase when it 
is combined with a policy like Prop 227 that moves EEs from bilingual to immersion programs 
(except for specification (7)). 

(please insert Tables 1-4 here) 

Table 3 displays the results for native English speakers (EO) math results. Generally, we 
find that EO students benefited in schools where there was a switch from bilingual education to 
immersion and an associated decrease in the number of EOs in their classroom (this is our 
interpretation of the coefficients on the interaction terms in the models). Reading test scores for 
EOs, presented in Table 4, also exhibited similar trends as math scores (although not always 
significant depending on the specification). 


Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

In summary, we find that the achievement scores of EEs declined by the switch from 
bilingual education to English immersion programs, with the exception of grades 2-3 reading 
scores. We also find that there is a beneficial effect of placing EEs into classrooms with more 
native English speakers. We also find that EOs were also affected by the implementation of 
Proposition 227. We find that EO students generally benefited from a school’s decision to switch 
EE students from bilingual education to immersion in combination with a decrease in the number 
of other EOs in their classroom. 


SREE Spring 2014 Conference Abstract Template 


4 



Appendices 

Not included in page count. 


Appendix A. References 

References are to be inAPA version 6 format. 

Chin, A., Daysal, N. M., & Imberman, S. A. (2012). Impact of bilingual education programs on 
limited English proficient students and their peers: Regression discontinuity evidence 
from Texas (No. wl8197). National Bureau of Economic Research. 

Ding, W., & Eehrer, S. E. (2007). Do peers affect student achievement in China's secondary 
schools? The Review of Economics and Statistics, 89{1), 300-312. 

Hanushek, E. A., Kain, J. E., Markman, J. M., & Rivkin, S. G. (2003). Does peer ability affect 
student achievement? Journal of Applied Econometrics, 18(5), 527-544. 

Hoxby, C. (2000). Peer effects in the classroom: Eearning from gender and race variation (No. 
w7867). National Bureau of Economic Research. 

Matsudaira, J. (2005). Sinking of Swimming? Evaluating the Impact of English Immersion 
versus Bilingual Education. Unpublished paper. University of California, Berkeley. 

Rossell, C.H., & Baker, K. (1996). The educational effectiveness of bilingual education. 
Research in the Teaching of English, 30(1), 1-1 A. 

Sacerdote, B. (2001). Peer effects with random assignment: Results for Dartmouth roommates. 
The Quarterly Journal of Economics, 116(2), 681-704. 

Slavin, R.E., & Cheung, A. (2005). A Synthesis of Research on Eanguage Reading Instruction 
for English Eanguage Eearners. Review of Educational Research, 75(2), 247-284. 

Slavin, R. E., Madden, N., Calderon, M., Chamberlain, A., & Hennessy, M. (2011). Reading and 
language outcomes of a multiyear randomized evaluation of transitional bilingual 
education. Educational Evaluation and Policy Analysis, 33(1), 47-58. 


SREE Fall 2013 Conference Abstract Template 


A-1 



Percent English Learners 


Appendix B. Tables and Figures 

Not included in page count. 


Figure 1 


Percent of Spanish ELs in Bilingual Education, 
LAUSD 

O J 
00 “ 




96-97 


97-98 


98-99 


99-00 


00-01 


School Year 


1 “ 

01-02 


SREE Spring 2014 Conference Abstract Template 


B-1 


Table 1: Regression Results for Math Scores (EL students) 


English Learners 


Grade 2-3 


Grade 4-6 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%m 

-0.001*** 

-0.002*** 

0 

0 

0.001 

0 

0 

0 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.001 

0 

0 







(0.00) 

(0.00) 

(0.00) 

fc Bil * % EO 






0 

-0.002 

0 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

95785 

95785 

95785 

95036 

95784 

95785 

95036 

95784 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%Bil 

0 

-0.001** 

0.001* 

0.002*** 

0.003*** 

0.001*** 

0.002*** 

0.003*** 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.002** 

0.001 

0 







(0.00) 

(0.00) 

(0.00) 

fc Bil * % EO 






-0.002 

-0.004** 

-0.001 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

68661 

68661 

68661 

68174 

68657 

68661 

68174 

68657 
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Table 2: Regression Results for Reading Scores (EL students) 


English Learners 


Grade 2-3 


Grade 4-6 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%Bil 

-0.004*** 

-0.005*** 

-0.002*** 

-0.001 

0 

-0.001*** 

-0.001 

0 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.003*** 

0 

0.002*** 







(0.00) 

(0.00) 

(0.00) 

% Bil * % EO 






0.005*** 

0.002* 

0.005*** 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

87056 

87056 

87056 

86267 

87054 

87056 

86267 

87054 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%Bil 

-0.001*** 

-0.002*** 

0.001** 

0.002*** 

0.003*** 

0.001*** 

0.002*** 

0.003*** 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.003*** 

0.002** 

0.002*** 







(0.00) 

(0.00) 

(0.00) 

% Bil * % EO 






0 

-0.004** 

0 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

65340 

65340 

65340 

64845 

65336 

65340 

64845 

65336 
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Table 3: Regression Results for Math Scores (EO students) 


English-Only 


Grade 2-3 


Grade 4-6 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 


No-IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

%Bil 

-0.014*** 

-0.014*** 

-0.001*** 

0 

0 

-0.001** 

-0.001 

-0.001* 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.001*** 

0 

-0.001* 







(0.00) 

(0.00) 

(0.00) 

% Bil * % EO 






0 

0.001 

0.002** 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

66339 

66339 

66339 

65496 

66339 

66339 

65496 

66339 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%Bil 

%EO 

% Bil * % EO 

-0.014*** 

(0.00) 

-0.014*** 

(0.00) 

-0.002*** 

(0.00) 

-0.001** 

(0.00) 

-0.001* 

(0.00) 

-0.002*** 

(0.00) 

-0.001 

(0.00) 

0.001 

(0.00) 

-0.002** 

(0.00) 

-0.001 

(0.00) 

0.002 

(0.00) 

-0.002*** 

(0.00) 

-0.003*** 

(0.00) 

0.003** 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

60761 

60761 

60761 

60191 

60761 

60761 

60191 

60761 
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Figure 4: Regression Results for Reading Scores (EO students) 


English-Only 


Grade 2-3 


Grade 4-6 



(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 


No-IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

w/IV 

%Bil 

-0.015*** 

-0.015*** 

-0.001*** 

-0.001 

0 

-0.001** 

-0.001 

-0.001** 


(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

(0.00) 

%EO 






0.002*** 

0.001 

0 







(0.00) 

(0.00) 

(0.00) 

?. Bil * % EO 






0 

0 

0.003** 







(0.00) 

(0.00) 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

64171 

64171 

64171 

63341 

64171 

64171 

63341 

64171 



(1) 

No-IV 

(2) 

w/IV 

(3) 

w/IV 

(4) 

w/IV 

(5) 

w/IV 

(6) 

w/IV 

(7) 

w/IV 

(8) 

w/IV 

%Bil 

%EO 

?> Bil * % EO 

-0.016*** 

(0.00) 

-0.016*** 

(0.00) 

-0.002*** 

(0.00) 

-0.001** 

(0.00) 

-0.001** 

(0.00) 

-0.002*** 

(0.00) 

-0.001 

(0.00) 

0 

(0.00) 

-0.002** 

(0.00) 

0 

(0.00) 

0.001 

(0.00) 

-0.002** 

(0.00) 

-0.002** 

(0.00) 

0.002 

(0.00) 

Class ctrls 



X 


X 

X 


X 

School ctrls 



X 

X 


X 

X 


Class EE 




X 



X 


School EE 





X 



X 

N 

59700 

59700 

59700 

59109 

59700 

59700 

59109 

59700 
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